n n 1 1 ii s iil mi mi in in mi ii m i m mi mi 



United States Patent [19] 

Hunziker 



US005910111A 
[ii] Patent Number: 
[45] Date of Patent: 



5,910,111 
Jun. 8, 1999 



[54] DISPLAY OF HEART MOTION 

[76] Inventor: Patrick Hunziker, 79 Curtis St., 
Somerville, Mass. 02144 

[21] Appl. No.: 08/941,150 
[22] Filed: Sep. 30, 1997 

[51] Int. CI. 6 A61B 5/05 

[52] U.S. CI 600/407; 359/23 

[58] Field of Search 600/407, 437; 

359/1, 22, 32, 33, 901; 73/603; 348/40; 

250/550; 356/347; 430A 

[56] References Cited 

U.S. PATENT DOCUMENTS 

3,889,226 6/1975 Hildebrand 340/5 H 

3,937,066 2/1976 Green et al 73/67.5 

3,937,555 2/1976 Benton . 

4,159,462 6/1979 Rocha et al 73/626 

4,337,992 7/1982 Johnson. 

4,415,225 11/1983 Benton et al. . 

4,498,729 2/1985 Benton . 

4,834,476 5/1989 Benton . 

5,038,791 8/1991 Collins et al. . 

5,098,176 3/1992 Wolf 359/11 

5,191,449 3/1993 Newswanger . 

5,245,448 9/1993 Waits. 

5,503,152 4/1996 Oakley et al 600/437 

5,519,517 5/1996 Redfield et al. . 

5,526,327 6/1996 Cordova, Jr. . 

5,543,25 1 8/1996 Taylor 430/1 

5,548,419 8/1996 Adrian et al. . 



5,553,618 9/1996 Suzuki et al 600/407 

5,592,313 1/1997 Hart. 

5,722,411 3/1998 Suzuki et al 600/439 

OTHER PUBLICATIONS 

Klug MA. A compact prototype one-step ultragram printer. 
SPIE Proceeding #1914, 'Practical Holography VIF, pp. 
15-19 (1993). 

Vannan MA. et al. Volumetric multiplexed transmission 
holography of the heart with echocardiographic data. J Am 
Soc Echocardiography 1995; 8: 567-575. 
Graham Saxby. Holography in Biology and Medicine. In: 
Graham Saxby, Practical Holography, 2nd edition, 1994, pp. 
429-435 Prentice-Hall, London. 

Primary Examiner — Brian Casler 

[57] ABSTRACT 

According to the invention, a method to process heart 
motion image data into a portable display medium is pro- 
vided. An image sequence of heart motion is provided. 
Image sequences are incorporated in angularly multiplexed 
holograms so that the parallax, i.e. the relative position of 
observer and holographic medium, encodes heart motion. 
This way, a change in the viewing angle of the printed 
hologram leads to the perception of the complete heart cycle. 
Using this technique, it is possible to incorporate a complete 
temporal image sequence of the beating human heart to a 
single printable image which can be used for display, 
storage, documentation and reporting. 

9 Claims, 2 Drawing Sheets 
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DISPLAY OF HEART MOTION 
FIELD OF THE INVENTION 

The present invention relates to a method of displaying 
heart motion, acquired by medical imaging techniques. 

BACKGROUND OF THE INVENTION 

Since prehistoric times, man has been able to capture and 
depict objects only in the form of static two-dimensional 
images. Only a century ago did it become possible to capture 
the motion of an object in the form of a series of single 
frames and to play them back as an animation. In 
echocardiography, motion was first represented in M-mode 
images, in which one axis of the display showed the echoes 
of a single scanline while a second axis coded time. The 
advent of two-dimensional echocardiographic imaging led 
to the current method of representing real time myocardial 
motion as video sequences. This development improved the 
understanding of cardiac structure in vivo and added new 
insights into cardiac function. However, viewing cardiac 
motion in a compact display becomes more cumbersome 
with 2D- compared to M-mode echocardiography. 

Currently, echocardiograms are stored primarily on 
videotapes, but digital storage media are being used more 
frequently for echocardiograms as well as for magnetic 
resonance and other medical imaging techniques. However, 
the search and review of studies on videotapes is time 
consuming and impractical, and both video and digital 
storage media require expensive and heavy equipment. 
Simple printing of single echocardiographic frames (similar 
to what is routinely done in radiology) is not practical 
because motion, a principal information content of heart 
imaging, is lost if a frame is printed. No methods exist to 
display, in a printable format, a moving heart in two- 
dimensional or three-dimensional images. At the current 
time, echocardiography machine manufacturers are devel- 
oping hardware and software that allow both the digital 
image acquisition and the presentation of echocardiographic 
studies on portable computers. Holography has been used to 
display static 3-dimensional images of the pelvis, the spine 
and the brain, by using computer tomography, magnetic 
resonance imaging and sonographic techniques to acquire 
image information. One description of holographic imaging 
of the heart was likewise limited to the representation of 
static images of a explanted animal heart. 

In contrast to the above described applications in 
bio medicine, dynamic datasets containing object motion, 
mainly of the heart, as acquired by ultrasound, computer 
tomography or magnetic resonance tomography, contain a 
significantly higher amount of data, posing additional prob- 
lems for representation in an understandable manner; appli- 
cation of holography to dynamic datasets of the beating 
heart has not been reported. 

SUMMARY OF THE INVENTION 

It is an object of the invention to display heart motion. 

It is another object of the invention to display heart 
motion on a portable medium. 

It is another object of the invention to process datasets 
acquired by medical imaging techniques, e.g. 
echocardiography, computer tomography and magnetic 
resonance imaging techniques, into a display format. 

It is another object of the invention to process heart 
motion datasets into a format suitable for storage and 
reporting. 
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It is another object of the invention to provide a display 
technique enabling a review of multiple heart motion loops 
at the same time. 

It is another object of the invention to provide a motion 
5 display in which motion can be slowed, stopped or reversed 
easily. 

According to the invention, a method to process heart 
motion image data into a portable display medium is pro- 
vided. Image datasets of heart motion are acquired by 

10 medical imaging techniques. The image datasets are 
exported from the acquisition machine in digital or analog 
format. After reformatting and postprocessing, the datasets 
are incorporated into angularly multiplexed holograms so 
that one parallax (i.e. the relative position of observer and 

15 holographic medium) encodes heart motion, thereby permit- 
ting the observer to perceive the motion of the complete 
heart cycle by tilting the hologram or changing his viewing 
position. 

Image sequences can be, e.g., two- or three- dimensional 
image loops acquired by echocardiography, computer 
tomography, magnetic resonance imaging or other feasible 
imaging techniques. The vertical or the lateral parallax of a 
hologram is used to encode time; the other parallax encodes 

25 the three-dimensionality of the heart if desired. Any holo- 
graphic production technique is applicable, provided it 
allows the introduction of temporally coded image 
sequences into the parallax of the resulting hologram. 
The above and other objects, features and advantages of 

30 the present invention will be better understood from the 
following specification when read in conjunction with the 
accompanying drawings. 

BRIEF DESCRIPTION OF THE DRAWINGS 

35 

Drawing 1 

Process of representation of heart motion by temporally 
coded, angularly multiplexed holography: 

1.1) A dataset is acquired by echocardiography. 

1.2) A series of consecutive frames is transferred to an 
imaging workstation. 

1.3) The datasets are reformatted, cropped and filtered. 

1.4) The dataset is then transferred to digital holographic 
45 printer or alternatively 

1.5) conventional optical holography is used to produce 
the final hologram (1.6). 

Drawing 2 

50 

Tilting of the resulting hologram (in this case, in vertical 
direction) leads to the perception of heart motion by the 
observer: 

2.1) The mitral heart valve (left sided inlet valve) is open. 
55 2.2) The mitral valve is closing. 
2.3) The mitral valve is closed. 

DETAILED DESCRIPTION 

6Q According to the invention, a method to display heart 
motion image data in a portable display medium is provided. 

Heart image sequences of human hearts or of experimen- 
tal animal hearts are acquired by ultrasound (transthoracic or 
transesophageal echocardiography), computer tomography, 

65 magnetic resonance imaging or other biomedical imaging 
techniques (Drawing 1.1). When using older standard clini- 
cal echocardiographic machines, digitization of the analog 
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video signal by a video digitizer was necessary. In contrast, ing time, because the two eyes of the observer have a 

we found it advantageous to use the latest generation different lateral viewing angle, but the same horizontal 

echocardiographic machines (e.g., the types Hewlett Pack- viewing angle relative to the hologram. When the horizontal 

ard Sonos 5500, Sonotron Vingmed System 5, Acouson parallax is used for motion display, it tends to confuse the 

Sequoia) which allows direct storage of the digitally 5 observer, because a different heart phase is seen by each eye. 

acquired image information, thereby avoiding the image The described invention makes it possible to reduce an 

degrading analog step. Thus, digital datasets consisting of a image loop of the beating human heart to a single "printable" 

series of 2D frames (or of 3D datasets) are produced (1.2), image which can be used for display, storage, documentation 

which represent consecutive timepoints of the heart cycle, and reporting, e.g. in echocardiography or other medical 

and which typically consist of 15-400 frames per beat in 2D 10 imaging techniques. 

motion imaging and of >800 frames per beat in 3D motion The ability of holography to represent multiple 
imaging. The conventional format for black-and-white dimensions, conventionally used to convey 3-dimensional 
images is 8 bit image depth with an image size of around space, is therefore expanded to display heart motion. Three- 
640*480 pixels image resolution, which was found to offer dimensionality can, in addition, be represented by encoding 
enough dynamic range for good results. Color images with 15 i t i n the other (usually lateral) parallax using computational 
a higher image depth, e.g. 24 bit, can also be used. holography. Any holographic printing method can be used to 
After the dataset is transferred to an imaging workstation print the hologram, if the method can substitute time- 
(both Compaq PC and a Macintosh PowerPC were used), it consecutive datasets for the customarily used multiple per- 
is converted to a non-proprietary format, if necessary; this spective views. 

was easiest for the cited Vingmed machine which exports the 20 Holograms can be exposed on different media, for 

datasets in a nonproprietary, easily readable format; for example, photopolymer foils (exemplified by the Polariod 

Hewlett Packard Sonos 2500, this is achieved by the manu- Mirage holographic film) or silver halide substrates. When 

facturer supplied software DSR2TIFF; for Acouson the hologram is printed as a reflection hologram (in contrast 

Sequoia, the Mountain View Software Package, supplied by t0 a transmission hologram, which offers less impressive 

the manufacturer, could be used. Then, spatial and/or tern- ^ results), a simple point-fight source is sufficient for optimal 

poral filtering may be applied (1.3): Best results were viewing of the hologram, making complicated, heavy or 

achieved by enhancing the image contrast, through the expensive viewing devices, like VCR and video screens 

histogram equalization algorithm or manual adjustment, superfluous. 

thereby presenting the cavity of the heart as mostly black, When tfae hol am is rinted on a ^ medi 

the heart valve tissue as maximum brightness, and the heart 30 oq a ^ Qf r ^ {{ ^ be ^ 

muscle tissue as intermediary grays. tQ reyiew a $tudy Qutside ^ medical Qr holograpMc imag . 

By assembling several (typically four) loops into the same ing laboratory, sent with a exam report, or used for 

dataset, side-by-side, it is possible to compare different archiving, both in the imaging lab or in a patients chart, 

views of the same heart or similar views of different hearts ^ ^ fc par t icu i ar i y useM because other methods to 

eas ^y* display, report and store heart motion images rely on heavy 

These postprocessed datasets are then transferred to a and expensive equipment which is usually not portable, 

workstation for production of a computational holographic By image seq uences is meant a series of datasets or 

stereogram, or alternatively optical holography can be used images displaying consecutive phases of the heart cycle, 

to produce a hologram. For hologram production, the 4Q Acquisition methods include rapid acquisition of one or of 

sequence information (time) is interpreted as a change in a few heartbeats, or electrocardiogram-triggered acquisition 

viewing angle m vertical parallax; i.e. subsequent images, of an cntire dynamic dataset over many heart beats. Image 

each with a time lag dt are dealt with as if dt represents a sequences can consist of series of digital datasets or of 

change in viewing angle alpha. For example, for a dataset of analog ( c g motion p ; cturc ) i mage s. 

30 consecutive time-consecutive frames, and a hologram Ae r „ „ • i j u-j- * i j , 

c _ - , ' , & . 45 Image sequences include multi-dime nsional datasets con- 

viewing range of 30 degrees, each consecutive frame is lainm 3^^^ and time information . To ^ or?OT ^ 

assigned an increasing viewing angle in steps of one degree. moving 3-dimeilsional datasets into a cardiac £ oiion 

From this "pseudo-spatial" dataset, an angularly multi- display, the dataset is first split into a time-sequential series 

plexed hologram is produced using standard hologram pro- 0 f stat i c 3-dimensional datasets. Then, each of these sets is 

duction techniques. Image sequences are incorporated in 50 volume rendered or surface rendered, using standard digital 

angularly multiplexed holograms so that the parallax, i.e. the imaging techniques. Holograms can be multiplexed in one 

relative position of observer and holographic medium, angular parallax or in two perpendicular parallaxes. For 

encodes time. In prior art, many methods are known which holograms which are multiplexed in only one parallax, one 

allow creation of a hologram using multiple frames. In perspective view of each of the mentioned time-sequential 

computational holography (1.4) the fringe pattern is 5S sta ti c 3D dataset is calculated by using standard digital 

computed, which would be produced by interference of a imaging techniques. The resulting frames are incorporated 

reference laser beam with a laser beam illuminating an i nt0 an angularly multiplexed hologram as described above, 

object. By contrast, a hologram is created in optical holog- Alternatively, moving 3D datasets can be used to produce 

raphy (1.5) by creation of the fringe pattern with actual use holograms which include time in one (usually vertical) 

of the laser light for object illumination and as reference 6Q para ilax and an additional spatial dimension in the other 

beam. The photosensitive medium is then exposed to this (usually lateral) parallax. To that end, multiple perspective 

fringe patterns to create the hologram. v i cws f or eacn time-sequential static 3D datasets are 

In summary, the perception of heart motion (time volume- or surface-rendered. These multiple perspective 

progression) in the resulting hologram is achieved by incor- views are used to multiplex the hologram horizontally. This 

pora ting the time sequence into a spatial parallax, which can 65 procedure is repeated with each consecutive static 3D 

then be viewed by tilting the hologram (Drawing 2). Best dataset in the vertical parallax. The resulting hologram will 

results are achieved by using the vertical parallax for encod- show a three-dimensional heart image when held still. When 
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the hologram is rotated laterally, the observer sees the heart The following non-limiting example further illustrates the 

rotating around its axis, and when the hologram is tilted present invention, 
vertically, the observer sees heart motion (contraction, valve 

closure etc). EXAMPLE 

Likewise, image sequences include representations of 5 

heart motion derived from the above mentioned sequences, Creation of a hologram representing a so-called 

which were, e.g., postprocessed to enhance or isolate bor- four chamber view of the human heart acquired by 

ders. echocardiography 

Image sequences can be monochrome or multicolor, and This example iUustra.es a typical proceeding to create a 

can include image sequences with color-coded velocity represe ntation of both heart muscle activity and heart valve 

information of cardiac tissue or uitracardiac blood, e g. ck)Sure ^ ; 

derived from Doppler techniques. Blood pool color Doppler 

allows the observer to visualize the temporal pattern of A volunteer underwent an echocardiography study, 

blood flow within the heart in a printable format. Ima g es were acquired using a Hewlett Packard Sonos 2500 

Each individual frame of a sequence may consist of 15 ^cartographic machine, using the apkal imaging win- 
multiple layers, e.g. overlay information from border detec- d ™ a " d t a standard four-chamber probe orientation. A 
tion algorithms standard clinical 2.5 MHz echocardiography transducer was 

. , , used. Electrocardiographic triggering was used for acquisi- 

The terms holography or hologram are meant to include Uon timi T lo 30 ; hea S bcat 

all techniques that integrate multiple image dimensions in a M weR , ifed and di ^ ^ Qn a t tical disk . 

flat, sohd display medium by means of a fringe pattern. <=» r 

Multiple techniques have been described in prior art to Image loops consisted of mdividual frames with a format 

create holograms by exposure of photosensitive substrates to of 640 * 480 P ixeis ™ d an depth of 8 bits in black and 

a fringe pattern. They include optical holography which uses white. 

object illumination and a reference laser beam to produce 25 The image loops were then transferred from the disk onto 

the interference pattern. Also included are holographic ste- a Compaq Personal Computer and reformatted into a format 

reography and other forms of computational holography, suitable for postprocessing. Using the NIH image software 

that compute the fringe pattern produced if object iltumina- on a Macintosh PowerPC 7200 computer, image contrast 

tion and reference laser beam were used. was enhanced and noise was filtered. Image loops were 

By holographic printer is meant any machine or process 30 cropped and enlarged to the desired format and resolution, 

which is capable of producing a hologram as defined above. Images were then transferred to a UNIX workstation. 

Another aspect of the invention is the review of heart Holographic reflection stereograms were then printed using 

motion in slow motion, still image or reverse time display, a one-step holographic stereogram printer, with an image 

which are easily achieved by tilting the hologram in the resolution of 640*480 pixels. Using vertical multiplexing, 

desired speed. This is especially useful for detailed study of 35 each of the time -sequential frames was incorporated into 

heart valve motion abnormalities or determination of increasing viewing angle of a hologram which was exposed 

regional and global heart muscle contractility. A further onto a holographic photopolymer foil, 

application ofthis invention is its use for comparing heart ^ re&m hologram disp i ayed , he left and me right 

muscle contractility and valve function during consecutive ventride> ^ ^ and (he trf a yalye and ^ ^ 

stages of heart stress testing. Durmg th K procedure, a < 0 when tilted, the motion of the heart was conveyed: the heart 

patient s heart is subjected to pharmacologic stimulation o mlisde first ^en relaxes, and the heart valve 

produce both increased heart rate and increased contract dosi ^ m flre dearl ^ ^ m d can 

velocity, thereby perm.ttog the diagnosis or exclusion of be V3jM or d in desjred itiQn tQ examine a 

coronary heart disease Whde the stress test results are stmcture of a motion m ^ ^ hasc rf ^ ^ 

conventionally described verbally, reviewed on videotapes 45 le Dimensions on me rint accurately reflect dimensions 

or on a computer m quad-screen format, the present inven- Qn ^ orf ^ ^ te ^ sb of 

tion is able to print several stages of the stress test side-by- *u u i -Z .-i * #u u <? *u • 1 

* « . - , , „ the hologram is similar to the sharpness of the original, 

side on a holographic printout, thereby allowing direct What claimed is- 

comparison of the contractility of the heart muscle in normal - . L . , - *, . . , „. 

j i * • J 1. A method for displaying a hologram of heart motion on 

speed or slow-motion. <n * i_i j- * • 

A -. ... o... . . , a portable medium, compnsing: 

A further practical application of this invention is the 

representation of moving three-dimensional heart images providing an image sequence from the beating heart; 

which may convey more easily understandable and more integrating the heart motion sequence into an angularly 

rapidly readable information than 2-dimensional images of multiplexed hologram so that one holographic parallax 

the heart. 55 encodes heart motion: and 

Former methods of producing holograms were very time displaying the hologram of the heart on a portable 

consuming, especially if done by a two step procedure with medium. 

the intermediate of a "master hologram". As a result, holo- 2. The method of claim 1 further including the step of 

grams were not widely used in clinical practice. With the acquiring the image sequence by one of echocardiography, 

development of holographic one step printers allowing pro- 60 magnetic resonance imaging, computer tomography, and 

duction of holograms based on digital datasets within szintigraphy. 

minutes, the current invention is applicable in clinical prac- 3. The method of claim 1 wherein said image sequence 

tice today. Those skilled in the art will be able to ascertain, depicts one of the whole heart, heart valves, and intracardiac 

using no more than routine experimentation, many equiva- blood flow. 

lents of the specific procedures of the invention described 65 4. The method of claim 1 wherein said image sequence 

herein. These and all other equivalents are intended to be comprises information derived from imaging one of heart 

encompassed by the claims made in this invention. tissue and intracardiac blood. 
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5. The method of claim 1 wherein said image sequence is 
a dataset of two or three spatial dimensions, including 
cross-sectional images, and including volume- and surface - 
rendered datasets. 

6. The method of claim 1 to include one of display, storage 
and reporting of heart motion. 

7. The method of claim 1 wherein said hologram is based 
on one of interference and diffraction. 
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8. The method of claim 1 wherein said hologram is based 
on one of optical holography and computational holography, 
and holographic stereograms. 

9. The method of claim 1 wherein said hologram is one of 
a transmission hologram and a reflection hologram. 
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